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ABSTRACT 



The purpose of this experiment was to assess the 
effectiveness of a comprehensive model for the analysis of hypermap 
navigation patterns through a comparison of navigation patterns associated 
with a traditional linear interface versus a non-linear "hypermap" interface. 
Twenty- six general psychology university students studied material on bipolar 
disorder, in which the home page was displayed in either an outline or 
hypermap format. The pages students went to, the time spent on each page, and 
the order of their traversal were all recorded. Cardinal (number of hits) , 
temporal (time on page) , and structural (pathfinder network) navigation 
measures were examined. The two groups did not differ on the cardinal 
navigation measure, but the temporal and structural measures differed 
markedly. Those in the hypermap groups spent significantly more time on the 
home page. Further, the structure of the navigation for those in the list 
group was more consistent with an outline than with a map structure, as would 
be expected. However, the navigation pattern of those in the hypermap group 
was no more consistent with an outline structure than it was with a map 
structure. The results indicate that a comprehensive analysis of navigation 
patterns can provide useful insight into hypermedia processing. Contains 19 
references. (Author/MES) 
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Abstract 

The purpose of this experiment was to assess the effectiveness of a comprehensive model for the 
analysis of hypermap navigation patterns through a comparison of navigation patterns associated 
with a traditional linear interface versus a non-liner “hypermap” interface. Twenty-six general 
psychology students studied material on bipolar disorder, in which the home page was displayed 
in either an outline or hypermap format. The pages students went to, the time spent on each 
page, and the order of their traversal were all recorded. Cardinal (number of hits), temporal 
(time on page), and structural (pathfinder network) navigation measures were examined. The 
two groups did not differ on the cardinal navigation measure, but the temporal and structural 
measures differed markedly. Those in the hypermap groups spent significantly more time on the 
home page. Further, the structure of the navigation for those in the list group was more 
consistent with an outline than with a map structure as would be expected. However, the 
navigation pattern of those in the hypermap group was no more consistent with an outline 
structure than it was with a map structure. The results indicate that a comprehensive analysis of 
navigation patterns can provide useful insight into h)^ermedia processing. 
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A Navigational Analysis of Linear and 
Non-Linear Hypermedia Interfaces 

Hypermedia-based instructional tools have become ubiquitous in classrooms at all stages 
of education in the last decade (e.g., Dillon & Gabbard, 1998; Kumar & Sherwood, 1997; 
Wilhelm, Friedmaim, & Erickson, 1998). This is most dramatically exemplified by the 
incredible popularity of the World Wide Web as an instructional tool (e.g., Khan, 1997; Owston, 
1997; Windschitl, 1998). Despite the widespread use of these methods, there has been a 
surprising lack of research on hypermedia tools, and as a consequence, researchers have called 
for more systematic research into the effectiveness of these approaches in general, and the 
conditions under which they are most effective (Dillon & Gabbard; Oweston, 1997; Jacobson, 
1994). In particular, there is a need for studies that examine the way in which students use these 
tools (Barab, Fajen, Kulikowich, & Young, 1996; Dillon & Gabbard, 1997). 

Hypermedia Navigation Measures 

The first purpose of this research was to carry out such an examination through the 
analysis of hypermedia navigation patterns. There are a number of potential advantages to such 
an approach. First, analysis of traversal through hypermedia is much less time consuming (Kelly 
& ODonnell, 1994) and intrusive (Barab, Fajen, et al., 1996; Kelly & ODonnell, 1994) than 
other methods of assessing on-going processing. Further, these approaches have been touted as 
examples of a measure that capture the dynamic nature of processing (Barab, Fajen, et al., 1996). 
Such techniques have great potential for use in research, instruction, and hypermedia design 
(Gay & Mazur, 1993). 

A number of experiments have been conducted, which have examined student's 
hypermedia traversal. For example, one approach is to classify pages in a given hypermedia 
space based on some criterion and then to count the number of times a user goes to a certain type 
of page (e.g., Kelly & O'Doimell, 1994; Lawless & Kulikowich, 1996; Barab, Bowdish, Young, 

& Owen). Another, less common, approach, is to analyze the order of the user's traversal 
through pages or the pattern of interconnection of links among the pages (Barab, Bowdish, et al., 
1996). The present project will extend this previous research by considering many of these 
approaches within a single context, within an overriding framework. We will take into account 
measures aimed at assessing the number of times a page is accessed (cardinal), the time on pages 
(temporal), and the pattern of linking (structural). It is our belief that such an approach will lead 
to an overriding m.odel of tracking behavior. Such a model-based collection of techniques would 
allow for the development of techniques that would be robust enough to apply across domains 
and flexible enough to be easily modified for a specific domain. 

In order to assess the effectiveness of this approach we will compare students' traversal 
through hypermedia pages that underlie two fundamentally different single page interfaces. If 
such a navigational approach is effective, the measures should differentiate between navigation 
patterns associated with the two interfaces, and should provide useful and insightful information 
about the nature of students' traversal. 
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HyperMaps 

In order to provide an ecologically valid examination of these navigation measures, we 
selected a "linear" interface in the form of a traditional outline, which is a way in which links to 
pages are often displayed on the World Wide Web (see Figure 1). As a contrast we created a 
non-linear HyperMap interface (Figure 2). This HyperMap interface is the product of the 
integration of a tool found effective in non-hypermedia experiments (Hall & ODonnell, 1996; 
Patterson, Dansereau, & Newborn, 1992; Rewey, Dansereau, & Peel, 1991), within the domain 
of hypermedia. Therefore, a secondary purpose of this experiment was to extend research on 
non-linear text displays, by assessing the effectiveness of such displays as interfaces for 
displaying a group of hyperlinked pages. 



Method 



Participants 

Twenty-six students in an undergraduate Physiological Psychology class at a medium 
sized Midwestern technologically oriented public imiversity participated in this experiment for 
extra class credit. 

Materials 

A HyperMap interface was developed to represent a description of the Bipolar I disorder, 
as presented at the Internet Mental Health Web Site (Long, 1998) (see Figure 2). In addition, a 
corresponding traditional linear interface was developed for the control group (Figure 1). Ten 
pages were selected from the site. Nodes on the HyperMap and a list of page titles for the list 
group were hyperlinks to these ten pages. Thus, the h)Apertext space for each group consisted of 
eleven pages including the main-interface page. 

In addition, a questionnaire consisting of nine Likert and an open-ended item, and a 
fifteen item multiple choice quiz were created to assess students subjective reactions and 
objective recall of the information. 

Procedure 

Participants were randomly divided into map and list groups. Students studied the 
information on the bipolar 1 disorder for thirty minutes, after which they completed the quiz 
followed by the questionnaire. The test and questionnaire were both forms on the World Wide 
Web. 



A WebTracking program was created for this experiment, which recorded the pages that 
students went to while studying and the time they arrived at and left a given page. 
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Results 



Navigation: Cardinal 

The analysis of tracking data began with a two-way mixed analysis of variance on page 
hits. Group (map vs. text) served as a between subject independent variable. Page (1 through 
11) served as a within subjects independent variable, and the number of times a student went to a 
page ("hits") served as the dependent variable. A main effect for Page was found F( 10,260) = 
\2.\\,p < .001. Post hoc analysis consisted of all possible comparisons among these pages 
collapsed across groups. The number of hits for the main-interface page was significantly higher 
than all other pages. The other pages did not significantly differ among themselves. No other 
effects were significant. 

Navigation: Temporal 

A two way. Group X Page, mixed analysis of variance was performed with average-time- 
on-page as the dependent variable. Again, a significant main effect for page was foimd 
F(10,250) = 26.47,/? <.001. Since a significant Group X Page interaction was also found F(10, 
250) = 4.47,/? < .001, a post hoc analysis of the main effect was not carried out. In order to 
further examine the interaction effect, Tukey's HSD tests were computed to compare groups on 
average time on page for each of the eleven pages. The descriptive statistics associated with this 
interaction, and the results of the post hoc tests are in Table 1. In addition, the means for the 
four pages on which the groups differed significantly are displayed in Figure 3. 

Navigation Structural 

Scoring. In order to create a structural navigation score for each participant, an 
adjacency matrix was created that represented the number of times that two pages had been 
"linked" during the given participant's traversal. Thus, all the pages were represented on both 
axes of the matrix and a given cell represented the intersection of two pages. The cell contained 
a 1 or a zero, depending on whether the pages were contiguous in a students navigation or not 
(Figure 4 is an example of how the adjacency matrix was created). Two prototypical adjacency 
matrices were also created, representing the prototypical map vs list structure. To create the list 
prototype, a given cell in the matrix received a one if the two pages were in the same group, and 
a zero if they were not. For the map prototype, the matrix was created in a similar manner, 
except that the grouping was different based on the structure of the map. Figures 5 and 6 
illustrates how the adjacency scores were created for these prototypes. For each participant, a 
map and list score were created by correlating the participant's matrix with the appropriate 
prototype, using the Knot Program (Interlink, 1999), based on Schvaneveldt's Pathfinder 
approach (Schvaneveldt, 1990). 

Analysis. A 2 X 2 repeated measures analysis of variance was computed with group (map 
vs list) as a between subjects independent variable and prototype (map vs list) as a within 
subjects independent variable. Prototype correlations served as the dependent variable. A main 
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effect for prototype was found F(l,26) = 3.81,/? < .05. The descriptive statistics for the map and 
list prototypes respectively were M= . 18, 5D = . 13 and M= .26 and SD= .\5. The group X 
prototype was also marginally significant F(l,26) = 3.81,/? = .06. (The means associated with 
this interaction are displayed in Figure 7). 

Group Comparisons on Quiz and Questionnaire Responses. 

This analysis began with a series of t-tests, comparing the two groups (map vs. text) on 
quiz score and on each of the nine Likert items on the questionnaire. The groups differed 
significantly in response to question #3 r(26) = 2.84,/? < .01; #5 r(26) = 2.31,/? < .05; and #6 
r(26) = 3.01,/? < .01. Those in the map group found the pages to be more helpful for learning 
the information {M= 6.38, SD = 2.84 vs. M= 3.80, SD = 1.94); in aiding them to remember 
information for the quiz (M= 4.77, SD = 2.65 vs. M= 2.77, SD = 2.02); and found the thirty 
minutes to be a more positive learning experience (M= 6.92, SD = 2. 1 8 vs. M= 4.80, SD = 

1.86). None of the other t-tests yielded statistically significant differences. 

An examination of students' responses to the open-ended question at the end of the 
questionnaire was consistent with the results of the Likert scale question analysis. In general 
students in the HyperMap group were more positive about the interface than students in the text 
group. Representative comments from students in both groups are displayed in Table 2. 

Discussion 

An examination of the analyses of the three tracking dimensions taken together, provide 
a rich insight into the nature of the differences in hypertext traversal in the map vs the list group. 
The cardinal analyses indicated that the groups did not significantly differ in terms of the 
number of times they visited the different pages. Both groups relied heavily on the interface 
page as an anchor point. Most likely the students returned to the page regularly as a gage of 
where they had been. However, the average-time-on-page analysis indicated that the groups 
differed markedly in how they actually used the interface page. Apparently, those in the list 
group spent very little time on the actual page. They simply went to it for some sort of 
orientation, and quickly moved to another page. They appeared to do the bulk of their actual 
studying on a few content pages (see Figure 3). However, those in the HyperMap group 
apparently used their interface, not only as an anchor, but also as a study aid, and as a graphical 
tool for the integration of the to-be-leamed information. 

The analysis of the navigation provided further insight into group differences. The group 
by prototype means (Figure 6) indicate that the structural navigation patterns for those in the list 
groups were much more consistent with their (list) prototype than with the map prototype as 
would be expected. However, the structural navigation pattern for those in the map group was 
no more consistent with the map prototype than for the list prototype. This finding adds further 
support to the explanation posed above, that those in the list group used their prototype more as a 
navigation aid, as opposed to a study tool. Taken together, these results provide support for the 
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utility of these navigation analyses as discriminating between different types of hypermedia 
designs, and providing insight into processing. 

The analysis of student quiz scores indicated that students did not significantly differ in 
tests over the content of the information studied. This finding is in marked contrast to the 
significant differences between the groups in their perception of the pedagogical effectiveness of 
the two different interfaces, as represented by questioimaire responses. These analyses indicated 
that those in the HyperMap groups found the interface to be substantially more effective. This 
inconsistency may be explained by the fact that the bulk of the information that students studied 
(i.e., the content pages) was the same for both groups. The questionnaire, on the other hand, was 
specifically aimed at comparing the interfaces. Further, the nature of the multiple-choice quiz 
consisted of specific questions, principally at the basic knowledge level, as most multiple choice 
tests do (Woolfolk, 1998). It is very possible that students were accurate in their subjective 
judgement of the effectiveness of the HyperMap interface, but the objective outcome measure 
used may have been insensitive to this effectiveness. 

Acknowledgement: This project was funded in part by a Grant from The American Psychology 
Association, Division 2, Teaching of Psychology, Instructional Research Award. 
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Table 1 

Average Time On Page (in seconds) as a flmction of Group and Page 



Group 

page HyperMap List 





M 


SD 


M 


SD 


Main/Interface Page* 


60.86 


26.42 


17.70 


11.90 


Major Depressed Episode* 


81.76 


38.77 


112.32 


57.78 


Hypomanic Episode* 


53.73 


18.54 


78.74 


47.72 


Manic Episode* 


54.70 


27.58 


94.88 


109.81 


Mixed Episode 


28.82 


11.15 


33.25 


34.70 


Single Manic Episode 


36.58 


15.46 


34.70 


17.44 


Most Recent Episode Depressed 


25.06 


19.53 


23.35 


8.66 


Most Recent Episode Hypomanic 


29.10 


26.68 


35.68 


29.10 


Most Recent Episode Manic 


67.46 


29.50 


58.67 


34.27 


Most Recent Episode Mixed 


70.31 


34.67 


90.06 


66.79 


Most Recent Episode Unspecified 


51.91 


37.86 


53.46 


29.17 



*significant mean difference (Tukey's Post Hoc Tests) 
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Table 2 

Representative Comments 



HyperMap Group 

• I foimd the main page effective because it gave an overview picture of how the different 
most recent episodes are linked to the different types of episodes. Being a picture helped me 
to remember it more. 

• The main page was very useful, I foimd that I would keep referring to it to get an overall 
view of the material presented. I thought it was useful, in that, I could return to it after each 
reading of the different sections and see it in context with the rest of the material. 

• I found the main page to be most helpful in reviewing the material. After I completed 
studying the pages, I returned to the main page to use it to test my recall of the information in 
the other pages. However, presenting the figure on the main page first thing was some what 
intimidating, but once I read the material the figure was much easier to follow and 
understand. 

• I did not initially understand the main page but after studying the written material and 
coming back to the diagram it proved to be very helpful. 

• I found the main page to be effective because it displayed an overview of the entire material. 

List Group 

1. I thought the main page was rather static and contained little information on the web site, 
where I was going, how to navigate. It is confusing to enter a web site that doesn't have any 
sort of site map that tells you where you are and where you're going 

2. I found the main page to be effective in giving me an outline of what was enclosed in the 
lesson. As far as helping me learn the material I don't think it helped me that much. 

3. The main page was all right as a sort of table of contents. The only reason I would go back 
to it though is because I had finished the bulk of my studying in 15 minutes and was just 
looking around verifying to myself that I knew all of the information. I think that the links 
within the text were more than enough for me to get by. 

4. The main page helped to organize a little bit but it still did not help to coimect the material. 

5. The main page was effective in outlining what topics were going to be studied. However, the 
main page did not relay any information about the topics and how the different topics related 
to each other. This fact made it more difficult to learn the material. 




12 



Navigational Analysis 
12 



• Diagnostic Criteria For Bipolar I Disorders 

o Bipolar I Disorder. Single Manic Episode 
o Bipolar I Disorder. Most Recent Episode Hvnomanic 
o Bipolar I Disorder. Most Recent Episode Manic 
o Bipolar I Disorder. Most Recent Episode Mixed 
o Bipolar I Disorder. Most Recent Episode Depressed 
o Bipolar I Disorder. Most Recent Episode Unspecified 

• Criteria For Mood Episodes 

o Mai or Depressive Episode 
o Manic Episode 
o Mixed Episode 
o Hvpomanic Episode 

Figure 1 . Linear (List) Interface 
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ERIC 



14 



Navigational Analysis 
14 




figure 3. Mean Time on P^e (second^) as a Function of Group 
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Order of Pag^s Accessed = 1 -2-4-3-4-1 



adjacency matrix 



Page \ 


2 


3 


4 


5 


1 


- 


1 


1 


0 


0 


2 


- 


- 


1 


1 


1 


3 


- 


- 


- 


0 


0 


4 


- 


- 


- 


- 


1 


5 


- 


- 


- 


- 


- 



f ^ure 4. Example of creation of adjacerKiy matrK from 
Older of Pages Accessed 



cells in adjacency 
matrix representing 
interactions AMONG 
these pages = 1 

cdls in adjacency 
matrix representing 
interactions AMONG 
these pages « 1 



• Diagnostic Ciiteria F or Bipolar I Disorders 
Bipolar 1 Disorder. Single Manic Episode 

o Bipolar 1 Disorder. Most Recent Episode Hvpamanic 
o Bipolar 1 Diairder. Most Recent E pigndffManif; 
o Bipolar I Diso rder. Most R ecent Episode Mxed 
o Bipolar 1 Disorder. Most Recent Episode Depressed 
Bipolar 1 Disorder. Most Recent Episode Unsnedfied^ 

• Ciitma F or Mood Episodes 

Major Depreave Episode '^ ^ , 

Manic Episode f ^ 

Mixed Episode : representing 

Hvpoipanic Episode m interaction BETWEEN pages 

in the different groups « 0 






Figure 5. Creation of List Prototype Atigaoency Matrix 
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cells in aojacency 
matrix representing 
: ^ interactbns AMONG 

= 1 

Dqarcssive 



* Manic . 

Episode .»* 



_ Bipolar 'I. ' •, 
Dif.ord« ^Aost:. 
h>cent Episode 
.UnrpecifierV ; • 




Cells in adjacency 
matrix representing 
interactbn BETWEEN pages 
in the different groups a 0 



ceils in adjacency 
matrix representing 
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these pages « ^ 



Table 6. Creation of hyperMap Prototype AdjacerKy Mathx 
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